1. Field
The subject invention relates to search systems and methods of searching using images. The subject invention also relates to systems and methods for image recognition and image-to-image comparison.
2. Related Art
Most websites today include digital images, such as digital photographs and videos. Most search engines include a feature that allows for searching of these digital images; however, these image search features use textual search queries that are compared with metadata for the digital images. Because some of the metadata is user generated, the metadata is sometimes inaccurate. Furthermore, some images have no metadata. Thus, the search results for the image search features tend to be unsatisfactory. In addition, the images are not ranked according to their level of similarity to the original query.
Content-based image retrieval (i.e., image-to-image comparisons) has been developed but has not been implemented in an online environment. Problems with the content-based image retrieval methods include the amount of time required to process the data and the accuracy of the comparison results. Some of the existing methods analyze the media content files (e.g., images, videos, etc.) for metadata to identify certain media patterns identical to those defined in preliminary descriptions, templates or objects (symbols, keywords, graphic patterns, audio phonemes). Some of these methods also recognize objects based on forms, contours, silhouettes, background, colors and other characteristics, or use graphical templates, such as “step”, “edge”, “roof”, “mountain range” or “lawn,” to recognize appropriate parts of the image. Other methods divide the image into logical units, apportion text and non-text areas, recognize objects, preliminarily allocate at least one graphical construction among smallest units as typical element for object recognition, and build object models as an aggregate of such typical elements with their spatial ratio. These methods, however, are limited to narrow domains (i.e., only recognition of text or simple geometric shapes). These methods also do not allow for classification, search and comparison of unknown objects, because the methods only work with the preliminarily defined types of objects and are limited to the components of the preliminary defined templates (e.g., phonemes for audio files, key frames for video files, graphical primitives for image files).
Other content-based image retrieval methods use file comparison algorithms. These methods find identical media-files, but do not analyze the content of these files. For example, some of these methods use fuzzy logic instruments that require significant computer resources and time and do not allow image searching of large arrays of data in real time. These methods are also limited in scalability because the time for search is linearly dependent on the number of files in the database.
Another method for identifying similar images develops a template based on a group of original images, using mathematical descriptors of forms and areas of the images in the group that are significant for search. The mathematical descriptors are developed based on parameters of the original image, which are compared with those in the database. A result is given as a number which is then compared with source parameters to determine the similarity of images. This method, however, still does not provide the speed of search necessary to conduct such searches in real time, because the descriptors for each group of initial images need to be changed every time a search is done, and the descriptor is then compared to all images in the database (i.e., calculate similarity parameters in the database and comparison of the source descriptor with them).
The most common method for indexing images is based on Fourier analysis. Fourier analysis gives quality results only based on contours of images. Attempts exist to improve search results using the results of Fourier analysis as input data for a neural network or by improving histograms of images by quantization based on frequency of color change. These attempts, however, have only been done for research purposes and only provide good search results in particular cases. The main problem with these approaches is the narrow domain of their potential use. For example, searches based on control points are used in face recognition systems to analyze images of human faces that have sufficient resolution and are in a particular perspective.
Other recognition methods separate images by their domain; then, different algorithms for mathematical descriptor calculation (index files) for the different domains are applied to the images to analyze the images. In other embodiments, a wide range of graphical primitives (knowledge base) are developed, and correlations between objects contained in a media file with the primitives in the database are determined. These methods, however, still do not identify the similarity of one media file to another because the methods can only be used with those media objects for which there are defined and reliable search algorithms and detailed databases of primitives. For example, because there are no reliable algorithms to search for trees, a search system using the known algorithms may identify a “tree” in the image, but cannot determine the similarity of the tree in the image to another tree. In addition, these methods have a high demand for processing computer resources to calculate the indexes of images.